
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Oflicc 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/764,633 



21127 



01/18/2001 



7590 02/06/2004 

KUDIRKA & JOBSE, LLP 
ONE state street 
SUITE 800 

BOSTON, MA 02109 



Nathan W. FuIIerton 



10040/7000 



8131 



EXAMINER 



BASOM, BLAINE T 



ART UNIT 



PAPER NUMBER 



2173 

DATE MAILED: 02/06/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application Nc 

09/764,633 



Examiner 

Blaine Basom 



Applicant(s) 

FULLERTON ET AL 



Art Unit 

2173 



" The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to comnnunication(s) filed on . 

2a)n This action is FINAL. 2b)13 This action is non-final. 

3) n Since this application is in condition for allowance except for fomnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-21 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) IE Claim(s) 1-21 is/are rejected. 

7) n Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 15 January 2002 is/are: a)|3 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d), 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orfonm PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (0- 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) 13 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received, 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 



Attachment(s) 

1) 13 Notice of References Cited (PTO-892) 

2) CH Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ information Disclosure Statement(s) (PTO-1449) Paper No(s) 4 . 



4) n Interview Summary (PTO-413) Paper No(s). 

5) □ Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 11-03) 



Office Action Summary 



Part of Paper No. 1 





Application/Control Number: 09/764,633 
Art Unit: 2173 



Page 2 



DETAILED ACTION 



Claim Objections 

Claim 17 objected to because the phrase, "each user-selectable regions," which is recited 
in the claim, is grammatically incorrect. Appropriate correction is required. 



The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 3, 10, 12-14, and 19 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Regarding claim 3, there is no antecedent basis for step "(f)>" 
which is expressed in the claim. In claims 10 and 14, there is no antecedent basis for "the 
presentation data." Also in claims 10 and 14, there is no antecedent basis for "other data 
sources," as there is no recitation of an original data source in the claims. By similar reasoning, 
"the source of data" recited in claims 12 and 13 lacks antecedent basis. In reference to claim 19, 
there is no antecedent basis for "the other data " 



Claim Rejections - 35 USC § 112 
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Claim Rejections - 35 USC §102 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 

basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1 (a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1 and 2 are rejected under 35 U.S.C. 102(e) as being anticipated by U.S. Patent 
No. 6,449,653, which is attributed to Klemets et al. (and hereafter referred to as "Klemets"). In 
general, Klemets teaches a method for delivering audio, video, and annotation data from a server 
to a client over a network. The client computer displays the audio and video data, in addition to 
the annotations, which describe the audio and video data (see abstract). 

Regarding claim 1, Klemets discloses that the client, which is described in the previous 
paragraph, may be implemented by a computer system having a display (see column 3, line 22 - 
column 4, line 42). Such a computer system is considered an apparatus, like that of claim 1 , 
which is for displaying content from a data file provided by a server. More specifically, Klemets 
discloses that this client computer comprises a "video/audio renderer" (see column 7, lines 9-18), 
which presents video and audio content from the data file provided by the server (see column 8, 
lines 6-9). This video/audio renderer is therefore considered a "media engine," like that recited 
in claim 1 . Additionally, it is further understood that this client computer, as described by 
Klemets, comprises: program logic for streaming content from the data file and for coordinating 
a presentation of the content by the media engine, the presentation having a plurality of data 
segments, each indexed by a corresponding time stamp (for example, see column 7, line 59 - 
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column 8, line 9; and column 9, lines 18-31); program logic for displaying an outline, 
specifically a "table of contents," which is displayed during the presentation (see column 7, lines 
9-45; column 5, lines 39-51; and figure 6); and program logic for accessing one of the plurality 
of data segments within the presentation upon selection of a corresponding portion of the outline 
of the presentation (see column 7, lines 1-8; and column 9, lines 18-36). 

Thus regarding claims 2 and 6, Klemets teaches a method implemented in such a client 
computer system, which has a display and is capable of generating a presentation from a stream 
of data, the method comprising: accessing the stream of data (see column 7, line 59 - column 8, 
line 5); extracting video and/or audio content from the stream of data and presenting this content 
on the display (for example, see column 8, lines 6-9); and extracting outline data representing a 
plurality of data segments within the presentation, whereby the data segments are linked to 
respective segments of the presentation, and whereby the outline data is presented on the display, 
as a table of contents, simultaneously with the presentation of the content data (see column 7, 
lines 1-45; column 5, lines 39-51; and figure 6). Specifically referring to claim 6, Klemets 
further teaches that this method may be implemented as program code, embodied on a computer- 
readable medium (see column 4, lines 25-35). Such a computer-readable medium having 
program code to implement the above-described method is considered a "computer program 
product," like that recited in claim 6. 

As per claim 9, Klemets discloses that, upon selection of a "content label" in the table of 
contents, i.e. outline, the link associated therewith is resolved and the audio and/or video content 
referenced by the link is extracted from the data stream and presented (see column 7, lines 1-8; 
and column 9, lines 18-36). As described in the previous paragraph such features may be 
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implemented by program code. Therefore, Klemets teaches: program code for, upon selection of 
a portion of the outHne data, resolving the link to the associated data segment within the audio 
and/or video presentation; and program code for presenting the content from the data stream 
associated with the selected segment. 

Referring to claims 10 and 14, Klemets teaches a method, implemented in a computer 
system having a display and being capable of generating a presentation from a stream of data, the 
method comprising: accessing the stream of data; extracting audio and/or video content from the 
stream of data; and presenting the content on the display, as is described above in the rejection 
for claims 2 and 6. Klemets further discloses that this method may also comprise extracting 
linking data representing a plurality of data segments within the audio/video content, whereby 
this linking data is displayed, via a table of contents, simultaneously with the presentation of the 
content (see column 7, lines 1-45; column 5, lines 39-51; and figure 6). In addition, this linking 
data may also represent at least one link to data other than the audio/video presentation data 
associated therewith, and may be linked to other data sources. For example, Klemets discloses 
that the labels in the table of contents not only link to specific segments of the audio/video 
stream, but also link to data segments in another data stream, specifically an "annotation stream" 
(see column 9, lines 18-36). Thus Klemets ftirther teaches extracting linking data representing at 
least one link to data other than the presentation data associated therewith, and presenting the 
linking data, via a table of contents, simultaneously with the presentation of the content data. 
Specifically referring to claim 14, Klemets fixrther teaches that this method may be implemented 
as program code, embodied on a computer-readable medium (see column 4, hnes 25-35). Such a 
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computer-readable medium having program code to implement this method is consequently 
considered a "computer program product," like that recited in claim 14. 

In regard to claim 1 1 , Klemets discloses that the user may pause the display of the 
audio/video content (see column 9, lines 8-17), and upon selection of the linking data in the table 
of contents, establish a link to the annotation data (see column 9, lines 18-36). 

As per claims 12 and 13, Klemets discloses that there are two types of annotation 
streams: a "locater annotation stream," whereby the data in the stream references annotation data 
from an external source, and a "data annotation stream," whereby the data in the stream is the 
annotation data itself (see column 5, lines 52-67; and column 6, lines 32-67). As described in the 
previous paragraph, upon selection of linking data in the table of contents, a link is established to 
an annotation stream in order to retrieve and display annotation data. Thus in the case of the 
stream being a data annotation stream, the source of the annotation data is the stream of data; in 
the case of the stream being a locater annotation stream, the source of the annotation data is 
external to the stream of data. 

With respect to claim 15, Klemets presents an apparatus for displaying content from a 
data file, the apparatus comprising: a media engine for presenting different media from the data 
file, such as audio and video; program logic for streaming content from the data file and for 
coordinating a presentation of the content by the media engine, the presentation having a 
plurality of data segments, each indexed by a corresponding time stamp; program logic for 
displaying an outline, specifically a table of contents, which is displayed during the presentation; 
and program logic for accessing one of the plurality of data segments within the presentation 
upon selection of a corresponding portion of the outline, as is described above in the paragraph 
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regarding claim 1 . In addition, the table of contents displays relevant links from the data stream 
to other data, as is described above in the rejection for claims 10 and 14, It is therefore 
understood that the apparatus of Klemets further comprises program logic for displaying relevant 
links from the data stream to other data, and program logic for resolving these relevant links to 
the other data. 

Conceming claim 16, Klemets teaches a method implemented in computer system, which 
has a display and is capable of generating a presentation from a stream of data, the method 
comprising: accessing the stream of data; extracting video and/or audio content from the stream 
of data and presenting this content on the display; and extracting outline data representing a 
plurality of data segments within the presentation, whereby the data segments are linked to 
respective segments of the presentation, and whereby the outline data is presented on the display, 
as a table of contents, simultaneously with the presentation of the content data, as is described 
above in the rejection for claim 2. This outline data is considered selection data, like that recited 
in claim 16, as it represents at least one user-selectable region, i.e. segment, within the 
presentation of the content data, the user selectable region associated with a command to display 
the region. Klemets discloses that, upon selection of such a user-selectable region represented in 
the table of contents, the presentation is modified to display the segment associated with the 
selection (see column 7, lines 1-8; and column 9, lines 18-36). Thus IGemets further teaches 
extracting selection data representing at least one user-selectable region within the presentation 
of the content data, the user selectable region associated with a command to display the region, 
and modifying the presentation of the content data upon selection of the user-selectable region 
associated with a selectable command. 
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As per claim 17, Klemets discloses that the table of contents may display data 
representing a plurality of user-selectable regions within the presentation (for example, see the 
table of contents, reference number 630, in figure 6). Klemets thus teaches extracting data 
representing a plurality of user-selectable regions within the presentation of the content data, and 
whereas described in the previous paragraph, each user selectable region is associated with a 
command. 

Regarding claim 18, Klemets discloses that, upon selection of a user-selectable region 
represented in the table of contents, the presentation is modified to display the segment 
associated with the selection (see column 7, lines 1-8; and column 9, lines 18-36). The user thus 
uses the table of contents to navigate through the presentation of the content data; the user moves 
to a segment of his or her choice in the presentation by using the user-selectable regions 
represented in the table of contents. 

As per claim 19, Klemets discloses that the user may pause the presentation of the 
audio/video content (see column 9, lines 8-17), and upon selection of the linking data in the table 
of contents, establish a link to another segment of the audio/video data (see column 7, lines 1-8; 
and column 9, lines 18-36). 

Regarding claim 21, Klemets presents an apparatus for displaying content from a data 
file, the apparatus comprising: a media engine for presenting content from the data file; program 
logic for streaming content from the data file and for coordinating a presentation of the content 
by the media engine, the presentation having a plurality of data segments; and program logic for 
displaying an outhne of the presentation during the presentation, as is described above in the 
rejection for claim 1 . In addition, the outhne includes relevant links from the data stream to 
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other data, as is described above in the rejection for claims 10 and 14. It is therefore understood 
that the presentation of the content has both a plurality of data segments and relevant links from 
the data stream to other data, and that the outUne includes both references to segments of the 
presentation and relevant links from the data stream to other data. 



Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 3-5 and 7-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
U.S. Patent of Klemets, which is described above, and also over U.S. Patent No. 6,278,446, 
which is attributed to Liou et al. (and hereafter referred to as "Liou"). As described above, 
Klemets teaches a method and computer program product, like that of claims 2 and 6, 
respectively, whereby outline data representing a plurality of segments within an audio/video 
stream is extracted and displayed as a table of contents. Klemets further discloses that, upon 
selection of a "content label" in this table of contents, a link associated therewith is resolved and 
the segment of audio and/or video content referenced by the link is extracted from the data 
stream and presented (see column 7, lines 1-8; and column 9, hnes 18-36). However, Klemets 
does not explicitly teach that the table of contents is displayed in a hierarchical arrangement, as 
is recited in claims 3 and 7, or that a position icon is generated and that the hierarchical 
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arrangement of the table of contents is traversed in correspondence with the presentation of the 
audio/video data, as is expressed in each of claims 4, and 8. 

Like Klemets, Liou presents a method for displaying a video, and navigating the video 
via a table of contents. Regarding the claimed invention, Liou discloses that this table of 
contents may be hierarchically organized, whereby each segment in the video is represented by 
an icon in the table of contents, the position of the icon in the table of contents corresponding to 
the position of the segment within the video (for example, see column 4, lines 55-67; and column 
13, line 52 - column 14, line 6), Liou thus teaches displaying a video table of contents, which is 
hierarchically organized, and generating a position icon and traversing the table of contents in 
correspondence with the presentation of the audio/video content. 

Consequently, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Klemets and Liou before him at the time the invention was made, to modify the 
table of contents taught by Klemets, such that it is hierarchically organized and such that each 
video segment is represented by an icon in the table, as is done by Liou. It would have been 
advantageous to one of ordinary skill to utilize such a combination because a hierarchically 
organized table of contents better depicts the video being presented, allowing for more efficient 
browsing of the video, as do icons representing the segments of the video, as is demonstrated by 
Liou. 

Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over the U.S. Patent of 
Klemets, which is described above, and also over U.S. Patent No. 6,570,587, which is attributed 
to Efrat et al. (and hereafter referred to as "Efrat")- As described above, Klemets teaches a 
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method, which is implemented in a computer system having a display and being capable of 
generating a presentation from a stream of data, the method comprising providing a data file 
containing a stream of data having intemal commands and user selectable options interleaved in 
the stream with the presentation data. For example, Klenlets discloses that a single data stream 
may comprise interleaved video, audio, and annotation data (see column 9, lines 37-46). The 
annotation data comprises event time markers, each marker corresponding to a command to 
retrieve and display annotation data from a specific source (see column 5, lines 52-67; column 6, 
lines 23-67; and column 8, lines 44-67), Additionally, it is understood that this interleaved 
stream comprises outline data representing a plurality of data segments within the presentation, 
whereby the data segments are linked to respective segments of the presentation, and whereby 
the outline data is presented on the display, as a table of contents, simultaneously with the 
presentation (see column 7, lines 1-45; column 5, lines 39-51; and figure 6). This outline thus 
comprises user selectable options for navigating to various segments of the presentation. 
Klemets fiirther teaches extracting the video and audio presentation data from the stream and 
generating a presentation thereof (for example, see column 8, lines 6-9), and also, extracting the 
annotation data, which corresponds to internal commands, from the data stream and interpreting 
the intemal commands to retrieve and display annotation data (for example, see column 8, lines 
44-58). Lastly, it is understood that the user selectable options are extracted from the data stream 
and presented via a table of contents, whereby the presentation is manipulated in response to user 
selection of one of these options (see column 7, lines 1-8; and column 9, lines 18-36). Klemets, 
however, does not explicitly teach superimposing such options for manipulating the presentation 
over the audio/video presentation, as is expressed in claim 20. 
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Like Klemets, Efrat discusses the presentation of multimedia data comprising video and 
audio. Regarding the claimed invention, Efrat teaches extracting user-selectable options from a 
video stream, and then superimposing these user-selectable options over the displayed video, the 
user-selectable options selectable to navigate to targets, such as other segments of the video, 
other videos, or HTML files (see column 3, line 30 - column 4, line 3). 

Therefore, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Klemets and Efrat before him at the time the invention was made, to modify the 
video stream taught by Klemets to include the user-selectable options taught by Efrat, which are 
superimposed over the displayed video. It would have been advantageous to one of ordinary 
skill to utilize such a combination because video may be a desirable means for presenting options 
to the user, as is taught by Efrat (for example, see column 1, lines 15-35). 



Conclusion 

The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. The appUcant is required under 37 C.F.R. § 1. 1 1 1(C) to 
consider these references fully when responding to this action. The Jain et al U.S. Patent cited 
therein presents a method wherein a video stream and an outline used to navigate the video 
stream are displayed simultaneously. Similarly, the Moeller et al. U.S. Patent described therein 
presents a system whereby a video stream and slider bar sued to navigate the video stream are 
displayed simultaneously. Lastly, the Mills et al. U.S. Patent cited therein describes a system for 
hierarchically organizing an outline of a video. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Blaine Basom whose telephone number is (703) 305-7694. The 
examiner can normally be reached on Monday through Friday, from 8:30 am to 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 746-7238. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 305-3900. 



btb 




